Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-35411: [MATLAB] Create a templated C++ Proxy Class for Numeric Arrays #35479

Merged
merged 22 commits into from
May 9, 2023

Conversation

sgilmore10
Copy link
Member

@sgilmore10 sgilmore10 commented May 8, 2023

Rationale for this change

This pull request is a followup to #34563. To facilitate the implementation of future array types, we would like to first create a C++ template class for numeric arrays. We also want to start adding basic tests for the array functionality in the MATLAB interface.

What changes are included in this PR?

  1. Added a C++ template Class called NumericArray templated on CType.
  2. Re-implemented the Float64Array C++ proxy class in terms of the new template class, i.e. NumericArray<double>.
  3. Added a method called double() on the MATLAB Float64Array class to convert the arrow.[Type]Array to a MATLAB double array.
  4. Added basic tests for round-tripping float64 arrays.
  5. Created a base C++ proxy Array class that all proxy array classes will inherit from.
  6. Renamed Print() to ToString() and made it return a string instead of printing to the screen.

Are these changes tested?

Yes, we added automated test cases to the test class tFloat64Array.m. In addition, we manually qualified these changes on macOS.

Are there any user-facing changes?

Yes, the Print() method is no longer public and there is now a method called double() on arrow.array.Float64Array.

Included below is a simple example of using the double() method:

>> arrowArray = arrow.array.Float64Array([1, 2, 3])            

arrowArray = 

[
  1,
  2,
  3
]

>> matlabArray = double(arrowArray)

matlabArray =

     1
     2
     3

>> class(arrowArray)

ans =

    'arrow.array.Float64Array'

>> class(matlabArray)

ans =

    'double'

Future Directions

  1. Support the rest of the numeric types.
  2. Add an abstract MATLAB base class called arrow.array.Array.
  3. Continue building out the methods (e.g. length())
  4. Support null values (validity bitmap).
  5. Handle converting non-ascii characters from UTF-8 to UTF-16.
  6. Handle errors in the C++ layer.

sgilmore10 and others added 14 commits May 4, 2023 10:06
classes for numeric arrays.

Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
arrays.

Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
…lmax, etc.)

2. Add DeepCopy parameterization to test points.

Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
@github-actions
Copy link

github-actions bot commented May 8, 2023

layer.

Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
matlab/src/cpp/arrow/matlab/array/proxy/numeric_array.h Outdated Show resolved Hide resolved
matlab/src/cpp/arrow/matlab/array/proxy/numeric_array.h Outdated Show resolved Hide resolved
matlab/src/cpp/arrow/matlab/array/proxy/numeric_array.h Outdated Show resolved Hide resolved
matlab/src/cpp/arrow/matlab/array/proxy/numeric_array.h Outdated Show resolved Hide resolved
matlab/src/matlab/+arrow/+array/Float64Array.m Outdated Show resolved Hide resolved
matlab/test/arrow/array/tFloat64Array.m Outdated Show resolved Hide resolved
matlab/test/arrow/array/tFloat64Array.m Outdated Show resolved Hide resolved
sgilmore10 and others added 7 commits May 9, 2023 10:06
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
@sgilmore10
Copy link
Member Author

Thanks for the feedback @kou! Let us know if you have any other comments.

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels May 9, 2023
Copy link
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@kou kou merged commit b372242 into apache:main May 9, 2023
9 checks passed
@github-actions github-actions bot added awaiting merge Awaiting merge and removed awaiting committer review Awaiting committer review labels May 9, 2023
@ursabot
Copy link

ursabot commented May 10, 2023

Benchmark runs are scheduled for baseline = 7f3e87d and contender = b372242. b372242 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️1.77% ⬆️0.06%] test-mac-arm
[Finished ⬇️0.26% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.93% ⬆️0.3%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] b372242b ec2-t3-xlarge-us-east-2
[Finished] b372242b test-mac-arm
[Finished] b372242b ursa-i9-9960x
[Finished] b372242b ursa-thinkcentre-m75q
[Finished] 7f3e87d2 ec2-t3-xlarge-us-east-2
[Finished] 7f3e87d2 test-mac-arm
[Finished] 7f3e87d2 ursa-i9-9960x
[Finished] 7f3e87d2 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@ursabot
Copy link

ursabot commented May 10, 2023

['Python', 'R'] benchmarks have high level of regressions.
test-mac-arm

liujiacheng777 pushed a commit to LoongArch-Python/arrow that referenced this pull request May 11, 2023
…ic Arrays (apache#35479)

### Rationale for this change

This pull request is a followup to apache#34563. To facilitate the implementation of future array types, we would like to first create a C++ template class for numeric arrays. We also want to start adding basic tests for the array functionality in the MATLAB interface. 

### What changes are included in this PR?

1. Added a C++ template Class called `NumericArray` templated on `CType`.
2. Re-implemented the `Float64Array` C++ proxy class in terms of the new template class, i.e. `NumericArray<double>`.
3. Added a method called `double()` on the MATLAB Float64Array class to convert the arrow.[Type]Array to a MATLAB `double` array.
4. Added basic tests for round-tripping float64 arrays.
5. Created a base C++ proxy `Array` class that all proxy array classes will inherit from.
6. Renamed `Print()` to `ToString()` and made it return a string instead of printing to the screen.

### Are these changes tested?

Yes, we added automated test cases to the test class `tFloat64Array.m`. In addition, we manually qualified these changes on macOS.

### Are there any user-facing changes?
Yes, the `Print()` method is no longer public and there is now a method called `double()` on `arrow.array.Float64Array`. 

Included below is a simple example of using the `double()` method:

```matlab
>> arrowArray = arrow.array.Float64Array([1, 2, 3])            

arrowArray = 

[
  1,
  2,
  3
]

>> matlabArray = double(arrowArray)

matlabArray =

     1
     2
     3

>> class(arrowArray)

ans =

    'arrow.array.Float64Array'

>> class(matlabArray)

ans =

    'double'
```

### Future Directions

1. Support the rest of the numeric types.
2. Add an abstract MATLAB base class called `arrow.array.Array`.
3. Continue building out the methods (e.g. `length()`)
4.  Support `null` values (validity bitmap).
5. Handle converting non-ascii characters from `UTF-8` to `UTF-16`.
6. Handle errors in the C++ layer. 

* Closes: apache#35411

Lead-authored-by: Sarah Gilmore <sgilmore@mathworks.com>
Co-authored-by: sgilmore10 <74676073+sgilmore10@users.noreply.github.com>
Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ArgusLi pushed a commit to Bit-Quill/arrow that referenced this pull request May 15, 2023
…ic Arrays (apache#35479)

### Rationale for this change

This pull request is a followup to apache#34563. To facilitate the implementation of future array types, we would like to first create a C++ template class for numeric arrays. We also want to start adding basic tests for the array functionality in the MATLAB interface. 

### What changes are included in this PR?

1. Added a C++ template Class called `NumericArray` templated on `CType`.
2. Re-implemented the `Float64Array` C++ proxy class in terms of the new template class, i.e. `NumericArray<double>`.
3. Added a method called `double()` on the MATLAB Float64Array class to convert the arrow.[Type]Array to a MATLAB `double` array.
4. Added basic tests for round-tripping float64 arrays.
5. Created a base C++ proxy `Array` class that all proxy array classes will inherit from.
6. Renamed `Print()` to `ToString()` and made it return a string instead of printing to the screen.

### Are these changes tested?

Yes, we added automated test cases to the test class `tFloat64Array.m`. In addition, we manually qualified these changes on macOS.

### Are there any user-facing changes?
Yes, the `Print()` method is no longer public and there is now a method called `double()` on `arrow.array.Float64Array`. 

Included below is a simple example of using the `double()` method:

```matlab
>> arrowArray = arrow.array.Float64Array([1, 2, 3])            

arrowArray = 

[
  1,
  2,
  3
]

>> matlabArray = double(arrowArray)

matlabArray =

     1
     2
     3

>> class(arrowArray)

ans =

    'arrow.array.Float64Array'

>> class(matlabArray)

ans =

    'double'
```

### Future Directions

1. Support the rest of the numeric types.
2. Add an abstract MATLAB base class called `arrow.array.Array`.
3. Continue building out the methods (e.g. `length()`)
4.  Support `null` values (validity bitmap).
5. Handle converting non-ascii characters from `UTF-8` to `UTF-16`.
6. Handle errors in the C++ layer. 

* Closes: apache#35411

Lead-authored-by: Sarah Gilmore <sgilmore@mathworks.com>
Co-authored-by: sgilmore10 <74676073+sgilmore10@users.noreply.github.com>
Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
rtpsw pushed a commit to rtpsw/arrow that referenced this pull request May 16, 2023
…ic Arrays (apache#35479)

### Rationale for this change

This pull request is a followup to apache#34563. To facilitate the implementation of future array types, we would like to first create a C++ template class for numeric arrays. We also want to start adding basic tests for the array functionality in the MATLAB interface. 

### What changes are included in this PR?

1. Added a C++ template Class called `NumericArray` templated on `CType`.
2. Re-implemented the `Float64Array` C++ proxy class in terms of the new template class, i.e. `NumericArray<double>`.
3. Added a method called `double()` on the MATLAB Float64Array class to convert the arrow.[Type]Array to a MATLAB `double` array.
4. Added basic tests for round-tripping float64 arrays.
5. Created a base C++ proxy `Array` class that all proxy array classes will inherit from.
6. Renamed `Print()` to `ToString()` and made it return a string instead of printing to the screen.

### Are these changes tested?

Yes, we added automated test cases to the test class `tFloat64Array.m`. In addition, we manually qualified these changes on macOS.

### Are there any user-facing changes?
Yes, the `Print()` method is no longer public and there is now a method called `double()` on `arrow.array.Float64Array`. 

Included below is a simple example of using the `double()` method:

```matlab
>> arrowArray = arrow.array.Float64Array([1, 2, 3])            

arrowArray = 

[
  1,
  2,
  3
]

>> matlabArray = double(arrowArray)

matlabArray =

     1
     2
     3

>> class(arrowArray)

ans =

    'arrow.array.Float64Array'

>> class(matlabArray)

ans =

    'double'
```

### Future Directions

1. Support the rest of the numeric types.
2. Add an abstract MATLAB base class called `arrow.array.Array`.
3. Continue building out the methods (e.g. `length()`)
4.  Support `null` values (validity bitmap).
5. Handle converting non-ascii characters from `UTF-8` to `UTF-16`.
6. Handle errors in the C++ layer. 

* Closes: apache#35411

Lead-authored-by: Sarah Gilmore <sgilmore@mathworks.com>
Co-authored-by: sgilmore10 <74676073+sgilmore10@users.noreply.github.com>
Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@sgilmore10 sgilmore10 deleted the GH-35411 branch June 27, 2023 23:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[MATLAB] Create a templated C++ Proxy Class for Numeric Arrays
3 participants